Automatic modelling and efficient tracking of deformable objects

This paper presents a system for efficiently tracking a deformable object in 3D. It is based on a model of the target represented as a set of textured features in 3D space and a set of shape bases, which encode the nonrigid modes of deformation. The model is constructed from a monocular sequence of 2D feature tracks using a non-rigid factorization algorithm followed by a non-linear optimization of the model parameters. Once the model is available, it is used for efficiently tracking the target using a 3D extension of the Inverse Compositional Algorithm which uses a projective camera model. In the experiments we show the performance of the system on synthetic and real sequences of a human face undergoing different facial expressions.